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Iterative Development of a Home Review 
Program to Promote Preschoolers’ 
Vocabulary Skills: Social Validity 
and Learning Outcomes 
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Purpose: This article describes the iterative development 
of a home review program designed to augment vocabulary 
instruction for young children (ages 4 and 5 years) 
occurring at school through the use of a home review 
component. 

Method: A pilot study followed by two experiments used 
adapted alternating treatment designs to compare the 
learning of academic words taught at school to words 
taught at school and reviewed at home. At school, children 
in small groups were taught academic words embedded in 
prerecorded storybooks for 6 weeks. Children were given 
materials such as stickers with review prompts (e.g., “Tell 
me what brave means”) to take home for half the words. 
Across iterations of the home intervention, the home review 
component was enhanced by promoting parent engagement 
and buy-in through in-person training, video modeling, 

and daily text message reminders. Visual analyses of single- 
subject graphs, multilevel modeling, and social validity 


measures were used to evaluate the additive effects and 
feasibility of the home review component. 

Results: Social validity results informed each iteration of the 
home program. The effects of the home program across sites 
were mixed, with only one site showing consistently strong 
effects. Superior learning was evident in the school + home 
review condition for families that reviewed words frequently 
at home. Although the home review program was effective 
in improving the vocabulary skills of many children, some 
families had considerable difficulty practicing vocabulary words. 
Conclusion: These studies highlight the importance of 
using social validity measures to inform iterative development 
of home interventions that promote feasible strategies for 
enhancing the home language environment. Further research 
is needed to identify strategies that stimulate facilitators and 
overcome barriers to implementation, especially in high-stress 
homes, to enrich the home language environments of more 
families. 


their future academic success and reading abilities, 

especially comprehension (Dickinson & Porche, 
2011; Stahl & Nagy, 2007; Tunmer & Chapman, 2012; 
Walker et al., 1994). Yet, systematic and explicit instruc- 
tion of sophisticated, academically relevant vocabulary is 
largely missing from preschool classrooms and from com- 
mercially available preschool curricula. Beck et al. (2013) 


Pp reschoolers’ vocabulary is a strong predictor of 
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recommend teaching Tier 2 vocabulary. We choose to call 
these words “academic vocabulary” as they typically are not 
used during conversational speech with children, but they 
have high educational utility and are likely to be encoun- 
tered in books (Beck & McKeown, 2007; Coyne et al., 2007). 
Although challenging, these words have the capacity to 
broaden the lexicon of young children, thereby improving 
the odds that they will be able to decode, recognize, and 
understand more words as they become readers (Vadasy 
et al., 2013). 

Young children enter kindergarten with vast differ- 
ences in their vocabulary knowledge (Hart & Risley, 1995; 
Hoff, 2003; Weisleder & Fernald, 2013). These differences 
place some children at greater risk for academic difficulties 
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(Cartmill et al., 2013; Dickinson & Porche, 2011; Hoff, 
2003; Suskind & Suskind, 2015). Differences in vocabulary 
exposure at home and at school are strong contributors to 
this persistent gap in word knowledge across groups of 
children (Walker et al., 1994). Providing timely, effective 
vocabulary intervention that supports word learning at 
school and at home can potentially promote the academic 
success of at-risk children. 


Bridging Vocabulary Instruction 
Between the School and Home 


Most of the existing vocabulary interventions for pre- 
schoolers are delivered either at home (Fielding-Barnsley & 
Purdie, 2003; Mol & Neuman, 2014; Price et al., 2009; 
Roberts, 2008; Whitehurst et al., 1994) or at school (Gonzalez 
et al., 2010; Neuman et al., 2011; Pollard-Durodola et al., 
2011; Wasik & Hindman, 2011). Few studies have investi- 
gated the potential benefit of providing at-risk preschoolers 
with vocabulary intervention that connects instruction 
between school and home settings (Lonigan & Whitehurst, 
1998; Whitehurst & Lonigan, 1998). Young children benefit 
from vocabulary exposure through adult interactions in 
the settings in which they spend most of their time (Dickinson 
& Tabors, 2001), namely, homes (Mol & Neuman, 2014; 
Price et al., 2009; Reese et al., 2010) and classrooms 
(Landry et al., 2009; Loftus-Rattan et al., 2016; Pollard- 
Durodola et al., 2011; Wasik & Hindman, 2011). 

The collective effort of teachers and caregivers using 
academic vocabulary during adult—child conversations 
could provide at-risk children with more word-learning 
opportunities (Neuman & Wright, 2014). Whitehurst and 
Lonigan (1998) found that vocabulary interventions com- 
bining instruction across the school and home settings for 
preschoolers yielded greater vocabulary gains than vo- 
cabulary instruction occurring only in classrooms. Aligning 
vocabulary instruction between school and home also has 
the potential of creating more consistent learning environ- 
ments for young children (Dickinson & Tabors, 2001; 
Hindman et al., 2016; Lonigan & Whitehurst, 1998). 

The most obvious barrier to successful home vocabu- 
lary intervention is lack of implementation (Mendez, 
2010). This barrier is likely to be more acute in low-income 
homes that experience limited financial resources, scarcity 
of time, and stress associated with daily routines in their 
home environments (Mendez, 2010; Shonkoff & Fisher, 
2013). These challenges could hinder caregivers’ participa- 
tion in home vocabulary interventions (K. E. Smith et al., 
2005; Teti et al., 2017). Indeed, these challenges often take 
priority in attempts to mitigate the effects of these risk 
factors (Halgunseth, 2009; Shonkoff & Fisher, 2013). Thus, 
it is important to consider how to minimize demands and 
maximize potential effects of parent involvement. 

Promoting parent engagement, setting realistic expec- 
tations, and increasing implementation at home requires 
collaboration between caregivers and professionals. For 
example, teachers and caregivers might employ strategies 
for maximizing two-way communication and involving 
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caregivers in decision making to build a partnership 
(Halgunseth, 2009; Halpern, 2000; Teti et al., 2017). Family 
engagement is likely to be maximized when tangible 
benefits (e.g., transportation and child care services, home 
education resources) and intangible benefits (e.g., environ- 
ments that welcome families) outweigh the costs of partici- 
pation (e.g., time and effort; Halgunseth, 2009). It also 
might be beneficial to provide parent education regarding 
their important role in promoting children’s vocabulary 
skills (Hart & Risley, 1995). 

To better understand what intervention components 
are perceived as feasible, valuable, and sustainable in au- 
thentic environments, we need to gather feedback from 
caregivers (Aldridge et al., 2016; Fixsen et al., 2013). One 
way of gathering, evaluating, and incorporating caregiver 
feedback is to measure social validity. Schwartz and Baer 
(1991) define social validity as “the use of evaluative feed- 
back from consumers to guide program planning and 
evaluation....” (p. 189). Although social validity is a vital 
component of external validity, it is often overlooked 
(Goldstein, 2016). Incorporating social validity measures 
can potentially improve the implementation of empirically 
supported practices in real-world settings (Eccles & Mittman, 
2006; Goldstein, 2016). Measuring social validity at all 
stages of development (i.e., before, during, and after) has 
the potential to enhance the feasibility, generality, and 
sustainability of interventions. Although social validity is 
important in developing evidence-base interventions, it is 
seldom reported in intervention studies (Goldstein et al., 
2014; Soto et al., 2019). 


Story Friends 


Story Friends is a supplemental vocabulary curricu- 
lum for preschoolers at risk for language and academic 
difficulties (Goldstein et al., 2016; Kelley et al., 2015; Spencer 
et al., 2012). This curriculum targets academic vocabulary 
(e.g., brave, gorgeous, tumble) through lessons embedded in 
storybooks. Academic, Tier 2 vocabulary is taught using 
the instructional procedures consistent with robust vocabu- 
lary instruction described by Beck et al. (2013). 

Story Friends teaches vocabulary skills through pre- 
recorded storybooks that contain embedded lessons. Story 
Friends is composed of two book series: Jungle Friends and 
Forest Friends. Each book series contains 13 books: one 
introductory book, nine instructional books, and three re- 
view books. Each book teaches two (Kelley et al., 2015) 
or four (Peters-Sanders et al., 2020) academic words. Em- 
bedded lessons provide multiple response opportunities, 
child-friendly definitions, examples and nonexamples, and 
text-to-life connections (Beck & McKeown, 2007). Each 
story takes about 10 min to complete. Story Friends books 
are delivered in small groups of three to four children who 
listen to the prerecorded stories under headphones. An adult 
facilitator ensures that children are on the right page and 
they are responding to the prompts in the book. Children 
hear the same book three times in a week before progres- 
sing to the next story. 
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The efficacy of Story Friends has been evaluated 
through a series of single-subject experimental design 
studies (Greenwood et al., 2016; Kelley & Goldstein, 2014; 
Kelley et al., 2015; Peters-Sanders et al., 2020; Seven et al., 
2020; Spencer et al., 2012) and cluster-randomized group 
design studies (Goldstein et al., 2016; Kelley et al., 2020, 
2015). Across studies, children identified with low vocabu- 
lary knowledge who received Story Friends instruction 
learned 30%-50% of the challenging words taught (Goldstein 
et al., 2016; Kelley et al., 2015; Spencer et al., 2012). This 
rate of learning tends to be higher than other studies seek- 
ing to explicitly teach academic words (see Kelley et al., 
2015). 

These promising findings prompted further develop- 
ment of Story Friends to increase children’s vocabulary 
learning. One way of strengthening the instruction provided 
in Story Friends is to provide additional opportunities for 
review and practice (Neuman & Wright, 2014). A home 
review component of Story Friends was developed because 
parents are often an underused resource for facilitating pre- 
schoolers’ vocabulary knowledge (Reese et al., 2010) and 
because there is a robust body of literature indicating that 
caregivers play a crucial role in young children’s vocabu- 
lary acquisition (Hart & Risley, 1995; Hoff, 2003, 2006; 
Pan et al., 2005; Weisleder & Fernald, 2013; Weizman & 
Snow, 2001). It was hypothesized that providing children 
with additional response opportunities at home would result 
in increased word learning. 


Piloting of a Home Review Component 
of Story Friends 


One way to encourage additional opportunities for 
practicing targeted vocabulary words is to have children 
recruit reinforcement through practice activities (Alber & 
Heward, 2000). Recruitment of reinforcement (Alber & 
Heward, 2000; Craft et al., 1998; Fowler et al., 2007; 
Wolford et al., 2001) is a strategy in which children are en- 
couraged to elicit positive adult attention. An example of 
recruitment of reinforcement is when a child is encouraged 
to call the positive attention of an adult when demonstrat- 
ing positive behavior by saying, “Look, I finished cleaning 
the toys!” (Craft et al., 1998). This seems to be a powerful 
and cost-effective way of facilitating children’s active in- 
volvement in their own learning while eliciting adult praise 
(Alber & Heward, 2000). 

This recruitment of reinforcement strategy served as 
the basis for a small-scale pilot study conducted to evaluate 
the feasibility of a home review component of Story Friends. 
After obtaining permission from the institutional review 
board (IRB), parents and children consented to participate 
in an initial pilot study. Six preschoolers (ages 4-5 years) 
listened to six Story Friends books containing embedded 
vocabulary instruction in small groups for 6 weeks at school. 
At the end of each week, the interventionists provided the 
children with stickers to take home containing prompts to 
recruit reinforcement, such as “Ask me what brave means.” 
Words on stickers were counterbalanced among the children. 


The interventionists sent home a note at the beginning of 
the pilot study explaining the home practice program to the 
parents, as well as weekly forms for the parents to indi- 
cate how often they practiced the target words at home. At 
the end of the review week, children were asked to define 
the targets words using a researcher-developed measure, 
but results did not reveal discernable effects of the home 
program on children’s word learning. 

Follow-up social validity interviews of parents helped 
explain the results. Although parents reported that they 
enjoyed receiving the stickers, they admitted rarely practic- 
ing the words at home. Some parents reported that they 
did not check their children’s backpack regularly as they 
received an abundance of miscellaneous papers each day. 
Parents offered a number of suggestions about better ways 
to remind them to practice the words. Although the re- 
search team postulated that sending home simple prompt- 
ing materials such as stickers was a good idea, it was clear 
that supplying stickers was not enough. 


Addressing Barriers to Improve 
the Home Review Program 


There are several factors potentially contributing to 
the null findings of this initial pilot study. First, parents 
were not taught to practice asking children to define words 
in person. Huebner and Meltzoff (2005) assert that parents 
benefit from in-person training. Perhaps not meeting the 
parents in person limited the researchers’ ability to estab- 
lish relationships with the parents, therefore reducing the 
level of parent buy-in and participation. However, when 
working with families, it is important to consider possible 
barriers (such as transportation difficulties, child care 
needs, food security, etc.) that may preclude parents from 
participating in a training program (Mendez, 2010). Second, 
the researchers sought to communicate with the parents 
using written correspondence (e.g., letters, parent diary 
forms), which was largely ineffective. Parents recommended 
communicating using technology they already use in their 
daily routines (e.g., text messages, social media). 

The social validity results of the initial pilot study 
played an important role in informing subsequent iterations 
of the home review program of Story Friends. Although 
reviewing Story Friends words initially seemed like it would 
be easy for parents to implement, the results of this study 
indicated that future iterations of this intervention needed 
to consider how parents were trained, to improve methods 
for maintaining school-home communication, and to 
consider barriers that might affect the extent of parent 
buy-in. 


Purpose 


We sought to identify successful approaches to 
bridging instruction between the school and home informed 
through social validity interviews. Simultaneously, we 
aimed to evaluate whether a home review program en- 
hanced vocabulary learning. This article describes two 
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experiments illustrating the iterative development of a 
home review component of Story Friends that incorpo- 
rated strategies to improve preschoolers’ vocabulary learn- 
ing by addressing potential barriers to implementation. Thus, 
a home review component of Story Friends was developed 
iteratively over two single-subject experimental design studies 
(see Table 1). 
The following research questions were addressed: 


1. To what extent do parent reports and social validity 
results attest to the feasibility of the home component 
of Story Friends? 


2. To what extent does the addition of a home review 
component to the Story Friends program improve 
preschoolers’ vocabulary learning? 


Experiment 1 (Site 1) 


In Experiment 1, we sought to increase parent engage- 
ment and participation. In addition to the stickers used in 
the initial pilot study, we added three additional strategies. 
First, we added text messaging. There has been growing 
evidence supporting the use of text messaging to enhance 
two-way communication between school and home settings 
(Bigelow et al., 2008; Carta et al., 2013; Kharbanda et al., 
2009). In the current study, we anticipated that text messaging 
would increase parents’ responsiveness and implementation 
of the home vocabulary program. 

Second, we provided training sessions to teach parents 
how to review the vocabulary words at home. The in-person 
training included the use of video modeling to demonstrate 
vocabulary-enhancing strategies caregivers could apply at 
home. Researchers have found that in-person trainings that 
incorporate video modeling are effective in enhancing care- 
giver participation (Bigelow & Lutzker, 1998; Briesch et al., 
2008; Huebner & Meltzoff, 2005; Jabaley et al., 2011; Lef- 
fel & Suskind, 2013; Mandel et al., 1998; Reamer et al., 


Table 1. Summary of iterative development of the home review program. 


1998; Reese et al., 2010). Jabaley et al. (2011) found that 
using video modeling technology through cell phones is an 
effective dissemination method. We hypothesized that 
video modeling would serve to instruct parents on how they 
could embed vocabulary instruction and prompt them to 
practice words in their daily routines. 

Third, we provided parents with options to over- 
come rigid work schedules, limited childcare, and lack 
of transportation (Castro et al., 2004; Mendez, 2010; S. 
Smith et al., 2013). Castro et al. (2004) note that parents’ 
employment is the strongest predictor of their level of in- 
volvement in their children’s early childhood education. 
Thus, we provided flexible times for trainings in their 
community. We hypothesized that providing parents 
with options that would mitigate potential participation 
barriers would bolster their implementation of the home 
review program. 


Participants 


Following IRB-approved procedures, parents and 
children were recruited from a School Readiness Prekinder- 
garten classroom in a low-income urban neighborhood. The 
classroom was part of a private childcare center offering 
financial subsidies to families through Florida’s Voluntary 
Prekindergarten program. In this early stage of research, the 
aim was to examine the effects of and engagement with the 
revised home review program under somewhat ideal condi- 
tions. Thus, we did not limit participation to only children 
with low oral language skills. Instead, children were selected 
to participate because their parents demonstrated an interest. 
Because the program was only available in English, children 
whose parents did not speak English fluently or children 
with language skills that fell more than two deviations below 
the mean on the standardized assessments administered were 
excluded. 

Table 2 presents a summary of participants’ demo- 
graphic characteristics. Six preschool children (two girls 


No. of 
words taught Prompting Home-school 
Iteration Measures per book materials Parent training communication 
Pilot study VMM 2 Stickers Sent home a letter describing Weekly written correspondence 
Parent diary form the intervention sent via children’s backpacks 
Social validity interview 
1_Site 1 Same as pilot study plus: 2 Stickers and In-person trainings containing Daily text reminders using free 
CELF Preschool-2 necklaces video modeling. Addressed messaging application, 
& PPVT-4 potential barriers to WhatsApp 
participation, including 
transportation, food, and 
child care 
2_Sites 2&3 Same as prior iteration plus 4 Stickers and Same as Iteration 1 Automated daily text reminders 
family demographic badges using free messaging 
questionnaire application, Remind 
Note. WMM = Vocabulary Mastery Monitor; CELF Preschool-2 = Clinical Evaluation of Language Fundamentals Preschool—Second Edition; 


PPVT-4 = Peabody Picture Vocabulary Test—Fourth Edition. 
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Table 2. Participant demographic information across sites. 


Primary Primary No. of times Hours a No. of times 
Age CELF caretakers’ caretakers’ per week stories week child a week child 
Iteration Child (years;months) Gender Race PPVT-4 Preschool-2 education marital status are read watches TV uses computer 
1_Site 1 Adriana 4:2 F African American 82 75 High school Single 
AJ 4;10 M African American 90 97 High school Single 
Vanessa 5;5 F African American 95 104 High school Single 
Jaiden 5;2 M Caucasian 88 104 Not reported Single 
Joshua 5;5 M African American 109 119 High school Single 
Marcus 4;10 M Multiracial 93 100 High school Married 
Average 4:8 92.83 99.83 
2_Site2 Odell 3;6 M Caucasian 119 79 Graduate Married 3 3-6 0 
Cynthia 43 F Caucasian 97 98 Bachelor Married 3 1-2 Daily 
Andrew 4;5 M Caucasian 99 84 Bachelor Widowed 3 1-2 0 
Lola 41 F Caucasian 117 108 Graduate Married 3 1-2 0 
Houston 4:5 M Caucasian 113 100 Bachelor Single 3 1-2 1-2x/week 
Tommy 4:7 M African American 97 100 Graduate Married 3 0 3x/week 
Alex 4;7 M Caucasian 117 106 Bachelor Married Daily 1-2 3x/week 
Annie 335 F Latina 109 88 Bachelor Married Daily 1-2 3x/week 
Average 4;2 108.5 95.34 
3_Site 3 Aliyah 4:2 F African American 90 88 High school Single Not reported Not reported Not reported 
Martin 4:2 M African American 88 71 Some college Single Daily 3-6 3x/week 
Collin 3;0 M African American 99 71 Some college Single Daily 3-6 3x/week 
Anthony 4:8 M African American 101 92 High school Single 3 3-6 1-2x/week 
Layla 43 F African American 911 84 High school Single Not reported Not reported Not reported 
Average 431 93.8 81.2 


Note. PPVT-4 = Peabody Picture Vocabulary Test—Fourth Edition; CELF Preschool-2 = Clinical Evaluation of Language Fundamentals Preschool—Second Edition; F = female; 
M = male. 


Downloaded from: https://pubs.asha.org Xigrid Soto on 03/11/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions 


and four boys; Mage = 4;8 [years;months]) and their parents 
participated. All of the parents had a high school educa- 
tion. Three of the primary caretakers were single mothers, 
one was a single father, one was a single grandmother who 
adopted the child, and one was a married couple. Their 
ages ranged from 24 to 60 years. Four of the children were 
African American, one child was multiracial, and one child 
was Caucasian. 

Children’s language skills were assessed using norm- 
referenced, standardized language measures to describe the 
participants. The Core Language Index from the Clinical 
Evaluation of Language Fundamentals Preschool—Second 
Edition (CELF Preschool-2; Semel et al., 2004) was used 
to evaluate children’s expressive and receptive language 
skills. The Core Index is composed of the Sentence Struc- 
ture, Word Structure, and Expressive Vocabulary subtests. 
The Peabody Picture Vocabulary Test—Fourth Edition 
(PPVT-4; Dunn & Dunn, 2014) was administered to evalu- 
ate children’s receptive vocabulary skills. The children had 
an average score of 92.8 (range: 82-109) on the CELF 
Preschool-2 and an average score of 99.8 (range: 75-119) 
on the PPVT-4, indicating that five of six children’s expres- 
sive and receptive language skills were within normal limits 
on the CELF Preschool-2 and the PPVT-4. 


Dependent Variables 


Parents’ Implementation of the Home Program 

Parent diary forms were used to measure how many 
times each word was practiced at home and to gauge the 
implementation fidelity of the home component. The parent 
diary forms were 4 x 6 in. rectangular cards containing 
12 preperforated squares and were designed to hang on 
refrigerators, doorknobs, or rearview mirrors of cars. There 
was a parent diary form for each target word. Parents were 
asked to keep track of how often they practiced each word 
by punching one of the boxes each time they reviewed a 
word at home. Parents were instructed to take a picture of 
the parent diary forms at the end of the week and to post 
it using a text messaging application. 


Social Validity 

A social validity interview was used to evaluate parents’ 
perceptions of the home review program. The social valid- 
ity interview contained questions related to how often 
parents practiced the words at home, parents’ feedback on 
the prompting materials used, whether their children used 
the target words spontaneously at home, parents’ perceptions 
of the feasibility of the home review program, whether they 
would continue practicing vocabulary words at home, and 
what suggestions they had for improving the program. The 
social validity interview took approximately 10-15 min to 
administer. The first two authors administered the social 
validity interviews over the phone. 


Children’s Vocabulary Learning 


A research-developed vocabulary assessment, the 
Vocabulary Mastery Monitor (Kelley et al., 2015), served 
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as the primary outcome measure of children’s vocabulary 
learning. This measure was used to evaluate the effects of 
the home review program. The Vocabulary Mastery Monitor 
asked children to define each target word (e.g., “What does 
brave mean?”). Children earned a score of 2 if they defined 
the word accurately, a score of | if they demonstrated partial 
knowledge of the target word, and a score of 0 for incorrect 
or nonresponses. The Vocabulary Mastery Monitor was 
administered before each child heard the book (pretest) and 
1 week after the review materials were sent home (posttest). 
Similar to previous studies, only two participants demon- 
strated full knowledge of one of the 24 target words (4.2%) 
at baseline; therefore, only the posttest scores were graphed 
to compare learning of words taught at school only versus 
words taught at school and at home. 


Procedure 


In-Person Parent Training 

Research assistants with experience in early childhood 
and/or speech-language pathology conducted the initial in- 
person meeting with the parents to inform them about the 
home program. The meeting was delivered in the commu- 
nity center located within walking distance of the preschool 
during a day and time that parents selected as most con- 
venient. Childcare and food were provided. Four of the six 
parents attended the meeting. The two parents who were not 
able to attend the meeting were provided the option to re- 
ceive the training in their homes. These at-home meetings 
were conducted | week after the initial group meeting. 

During the in-person training, parents who decided 
to participate in the program completed a consent form and 
a program preferences survey. The survey solicited feed- 
back on preferences on the type of review materials (i.e., 
stickers or necklaces) they wanted and when to receive daily 
text reminders to practice the words via WhatsApp, a free 
text messaging application. 

A PowerPoint presentation contained a video dem- 
onstrating home vocabulary practice. In the video, a par- 
ent asked a child to define a word, modeled how to use the 
word in a sentence, and asked the child questions related 
to the word (e.g., “How do you feel when you go to the 
dentist? Are you brave? What does brave mean?”). The pre- 
sentation also included an explanation of why vocabulary 
is important to children’s development, an overview of the 
Story Friends program, and steps for reviewing the target 
words at home. Parents also were encouraged to tabulate 
how often they practiced words at home by using the parent 
diary form. Parents were asked to send pictures of their 
parent diary forms at the end of each week. Data gathered 
from these pictures were collected to estimate the frequency 
of word practice at home. The training took approximately 
20 min to complete. 


Initial Assessments 

Once parents provided informed consent, children’s 
expressive and receptive language skills were evaluated 
using the PPVT-4 and the CELF Preschool-2. These 
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assessments were conducted in quiet rooms adjacent to 
classrooms to avoid distractions. Once all the children’s 
language skills were assessed, six children were assigned to 
one of two small groups. 


Small Group Listening Centers 

At school, children received vocabulary instruction 
through the Story Friends curriculum in small groups. Chil- 
dren heard the prerecorded stories using headphones. Lis- 
tening sessions took approximately 10 min to complete and 
were repeated three times per week for a total of 6 weeks. 
Each book taught two academic vocabulary words. Vocab- 
ulary instruction included the use of child-friendly definitions, 
the use of real-world examples, multiple exposures, and 
multiple response opportunities. As children listened to the 
storybooks, an adult facilitator managed the group by ensur- 
ing children were on the right page and engaged during the 
intervention. 


Prompts for Home Vocabulary Practice 

At the end of each week, children were given stickers 
or necklaces with review prompts designed to recruit addi- 
tional practice opportunities for one of the two words tar- 
geted at school (e.g., “Ask me what gorgeous means?”’). 
Children were encouraged to keep their stickers or neck- 
laces on all day. Per teacher recommendations, some chil- 
dren’s stickers were placed on their backs to ensure they 
did not lose their stickers before getting home. 

After stickers were sent home, the interventionists 
sent daily text reminders saying, “Don’t forget to practice 
your child’s words today.” The text messages were sent 
using WhatsApp. These text messages were sent in the eve- 
ning, as per parents’ preferences. 


Parent Assessments 

Parents were interviewed using the social validity 
interview at the end of the study to gather information 
about their perceptions of the home review program. 


Experimental Design and Analysis 


The effects of the home review component was evalu- 
ated using an adapted alternating treatment design, a single- 
case experimental design useful for comparing experimental 
conditions applied to nonreversible behaviors (Ledford & 
Gast, 2018). For each book, one word received school-only 
instruction in listening centers and one word received 
school instruction plus home review (with review prompt- 
ing materials sent home). Thus, learning of the school-only 
words versus the school + home words provided an indica- 
tion of the effects of the home review strategies. Counter- 
balancing of words across children minimized the potential 
confound related to systematic differences in the difficulty 
in words. 

To evaluate the social validity of the home review 
component, parents’ responses from the parent diary forms 
were summarized to calculate how often they reported that 
words were practiced at home each week. The average rate 


of parents’ use of the parent diary forms and the mean 
number of times each word was practiced were recorded. 
For the social validity interviews, parents’ responses were 
first transcribed. Their responses were then summarized to 
determine their general perceptions of various aspects of 
the home program (i.e., in-person training, video modeling, 
prompting materials). 

To evaluate learning outcomes across conditions, 
children’s posttests scores on the Vocabulary Mastery 
Monitor were graphed. The graphs were visually inspected 
to determine whether gains in vocabulary knowledge were 
consistently greater for the school + home words com- 
pared to the school-only words. The extent of replications 
of this effect within and across participants allowed us to 
determine the extent to which experimental effects were 
demonstrated. 

Statistical analyses were conducted to corroborate 
the findings of the visual analyses. A two-level random 
coefficients model was used to evaluate children’s average 
vocabulary knowledge. Multilevel modeling (MLM) was 
applied to estimate the extent to which the home review 
component influenced the variation in vocabulary knowl- 
edge of the children in three preschool sites. This analysis 
enabled an estimate of the differences in the average vo- 
cabulary knowledge of children for each condition: school- 
only versus home + school. The parameter estimates for 
vocabulary knowledge range from 0 to 2 for each word. 
The model was applied separately for each site by using 
SPSS MIXED models with restricted maximum likelihood 
estimation. A Satterthwaite estimated degrees of freedom 
was calculated to ensure there were accurate inferences 
from the single-subject data (Ferron et al., 2009, 2010). 
To disentangle the vocabulary knowledge variation of 
children in the school-only versus school + home conditions, 
the following two-level random coefficients model was 
specified: 


WK, = oj + 1; Treatment + r;(Level 1) 
To; = Boo + uoj(Level 2) . () 
1; = Byo(Level 2) 


The WK; represented ith vocabulary knowledge for 
the jth child; treatment was coded | for the school + home 
condition and 0 for the school-only condition. Thus, Boo 
represented the average vocabulary knowledge per word 
among children in the classroom for the school-only condi- 
tion, and {jo indicated the average vocabulary knowledge 
per word between the school + home and school-only con- 
ditions. The error terms (e.g., 7;, Uj) were assumed to be 
normally distributed. 


Fidelity and Reliability 


Implementation fidelity was evaluated using a proce- 
dural observation checklist on 20% of the small group ses- 
sions. Researchers scored fidelity on each other using the 
observation checklist. The checklist included procedures 
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that were vital for lesson delivery, including ensuring chil- 
dren had the materials necessary to complete the listening 
center, not interrupting the audio, maintaining children’s 
engagement in the storybooks, and distributing home 
materials. The percentage of implementation fidelity was 
calculated by dividing the total number of observed proce- 
dural points by the total number of possible procedural 
points. Implementation fidelity averaged 99% (range: 94%-— 
100%). 

The research staff were taught to administer the 
research-developed and standardized language measures with 
high implementation fidelity. The administration of the 
Vocabulary Mastery Monitor was audio-recorded to evalu- 
ate fidelity of administration and scoring reliability. A 
second research staff listened to 20% of all testing sessions 
to judge whether the assessment protocol was delivered 
with high fidelity and whether responses were accurately 
recorded. The fidelity of assessment administration was 
100%. 

After the first two authors scored all measures, a 
second trained research member blind to which words were 
in each condition independently scored 20% of the Vocab- 
ulary Mastery Monitors to assess reliability. Item-by-item 
response agreement was calculated by dividing the agree- 
ments by the total number of disagreements and agreements. 
The interrater reliability score was 100%. 


Results 


Parent Implementation and Social Validity 
of the Home Component 


Parents’ Implementation 

Researchers suggested that parents practice the words 
at least 12 times a week. The parent diary forms were 
collected to estimate how often parents reported practicing 
the words at home as a measure of implementation fidelity. 
On average, the parents returned 69% of the parent diary 
forms (range: 0%-100%). The number of times the words 
were practiced at home ranged from five to 13 times a 
week. 


Social Validity 

Five of the six parents answered the social validity 
questions over the phone; one parent did not respond de- 
spite multiple attempts. All of the parents interviewed 
expressed that they saw the value of the home review pro- 
gram but that their busy schedules sometimes interfered 
with their implementation of the home program. 

Parents reported that the in-person parent training 
was helpful and that they appreciated the flexibility in 
scheduling. Parents stated that they benefitted from video 
modeling and seeing how to provide additional practice 
opportunities for word learning at home. 

When parents were asked how often they practiced 
the words at home, they reported they practiced the words 
at home and in the car every day; however, they did not 
always have the time to send the parent diary forms over 
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the phone. One parent said, “I am busy because I got a 
new job, and I have not been good about the chart [parent 
diary form]. But I keep an estimate in my mind of how 
often we practiced. For example, for ridiculous, I have 
practiced six times.” Overall, parents reported that they 
enjoyed receiving daily text reminders to practice the words 
through WhatsApp. However, one parent noted concern 
regarding the use of a phone application because other 
people outside the study could view her profile. In gen- 
eral, data gathered from the interviews indicated that, 
although they enjoyed various aspects of the home pro- 
gram, there was variability in their implementation of the 
home program. 


Differential Effects on Children’s Word Learning 


Vocabulary learning results are displayed in Figure 1. 
Graphs are arranged by the estimated order of home im- 
plementation (from highest to lowest starting from the top 
left and progressing to the bottom right). The level of 
home implementation was estimated based on how often 
parents reported practicing the words at home through 
the parent diary forms. Pretest scores on the Vocabulary 
Mastery Monitor indicated negligible knowledge of words 
prior to the intervention, with the exception of Joshua who 
knew three of 12 words at pretest. The x-axis represents 
each book, and the y-axis presents children’s performance 
on the posttest Vocabulary Mastery Monitor probes (a 
week after school instruction). The solid black lines with 
the black circles show children’s posttest scores on the 
school-only words, and the dashed black lines with the 
open squares show posttest scores on words in the school + 
home condition. We hypothesized that the dashed line 
would be higher than the solid line, indicating that the 
home review component resulted in better vocabulary 
learning. 

Although all six children demonstrated learning of 
academic vocabulary words, visual inspection revealed no 
clear advantage for the school + home words. Ceiling 
effects were demonstrated by AJ and Joshua, who learned 
almost all the words regardless of condition. Marcus showed 
an advantage for three of five school + home words, but 
an advantage for one of the school-only words. The other 
three children (Vanessa, Adriana, and Jaiden) showed 
mixed effects. The scores averaged 66.7% and 58.3% for 
Adriana, 83.3% and 66.6% for Vanessa, 80% and 50% 
for Marcus, 100% and 91.7% for Joshua, 83.3% and 91.7% 
for AJ, and 50% and 50% for Jaiden, for the home + school 
versus school-only conditions, respectively. 

The MLM results (summarized in Table 3) supported 
the visual analysis of the graphs. Evaluating model assump- 
tions for each of the residuals from Site 1, we confirmed 
the linearity, homoscedasticity, and normality assumptions 
by using a boxplot, a scatter plot, and a Q-Q plot. The re- 
sults of the MLM analysis revealed that children learned 
an estimated average of 1.23 word points for the school- 
only condition and 1.53 word points for the school + home 
condition. The difference of 0.29 word points was not a 
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Figure 1. Experiment 1 results (Site 1). Children’s posttest scores on the Vocabulary Mastery Monitor for each book. Open squares denote 
the home + school condition, and closed circles denote the school-only condition. Graphs are arranged by the level of home implementation 
as indicated by parent diary forms (highest to lowest, starting from the top left graph and progressing to the right). 


Adriana 
G- a 


Posttest score on Vocabulary Mastery Monitor 


Vanessa 
as - aie 


Joshua 


statistically significant difference, 7(61.3) = 1.40, p = .166. 
As can be seen in Table 3, we also observed significant 
within-child variation for the six words per condition among 
Site 1 children (6* = 0.75). 


Discussion 


The parent diary forms provided some evidence that 
most of the parents reviewed the target words at home. 
The social validity results indicated that parents enjoyed 
and valued the program. Although children did not dem- 
onstrate consistent enhancements in word learning associ- 
ated with the home review component, data gathered from 
social validity and parent diary forms revealed that the 
level of parent engagement was greatly improved over the 
pilot study. Many parents in this experiment shared the 
parent diary forms weekly by sending pictures electronically. 
This approach seemed more effective than asking parents 
to return paper-based materials to school. The findings 
from this study support the utility of applying technology 
to promote parent engagement. Engaging with parents 


using cell phones and social media (e.g., Whats App, 
Facebook, Instagram) is a promising way of improving 
parent responsiveness. 

The social validity data provided further information 
on the acceptability of several aspects of the home review 
program. First, providing in-person training seemed to 
facilitate the establishment of a trusting collaboration 
between the parents and the researchers. This was evident 
as parents were willing to share their feedback on the home 
program. Involving parents in the development of home 
programs seemed beneficial, as it emphasized the message 
that parents are important stakeholders in their children’s 
learning and that their opinions matter (Teti et al., 2017). 
These findings are in line with research indicating that one 
of the keys to a successful home intervention is establishing 
trusting, reciprocal relationships between the school and 
home (Halgunseth, 2009). Other investigators have noted 
that in-person parent trainings alone are rarely sufficient to 
increase parent engagement (Huebner & Meltzoff, 2005). 
However, text messaging helped to sustain relationship 
development. 
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Table 3. Output of mixed-effects models for estimating children’s vocabulary knowledge for each site. 


Parameter Parameter estimate SE 95% Cl Significance 
Site 1 fixed effects 

Intercept 1.233 0.177 [0.85, 1.62] < .001 

Condition 0.294 0.210 [-0.13, 0.71] 166 
Site 1 variance estimates 

Within-child 0.750 0.135 [0.53, 1.07] < .001 

Between-child 0.537 0.748 [0.004, 1.07] 473 
Site 2 fixed effects 

Intercept 0.662 0.142 [0.37, 0.95] < .001 

Condition 0.777 0.126 [0.53, 1.03] < .001 
Site 2 variance estimates 

Within-child 0.750 0.080 [0.61, 0.92] < .001 

Between-children 0.763 0.577 [0.02, 0.34] -186 
Site 3 fixed effects 

Intercept 0.567 0.195 [0.08, 1.05] .028 

Condition 0.086 0.159 [-0.23, 0.40] 543 
Site 3 variance estimates 

Within-child 0.730 0.098 [0.56, 0.95] < .001 

Between-children 0.127 0.111 [0.02, 0.71] .254 


Note. Cl = confidence interval. 


We found that text messaging played unanticipated 
roles beyond simply prompting parents to practice words 
at home. Two parents spontaneously shared pictures and 
videos of their children practicing the target vocabulary 
words at home. Researchers in turn reinforced parents’ 
engagement, making parents and researchers equal players 
in children’s word learning. Finally, text messaging 
allowed for networking among parents. Parents communi- 
cated with each other, exchanging ideas and encouraging 
one another’s participation in the home program. These 
findings provide preliminary evidence of ancillary benefits 
of establishing school-home communication using text 
messaging. 

Text messaging seems advantageous for several 
reasons. First, cell phones and text messaging have become 
commonplace in American society. Thus, sending daily 
text reminders fits well within parents’ existing routines. 
Identifying ways of embedding home practice opportunities 
that fit within parents’ daily routine may increase their 
level of engagement. Second, unlike notes sent home, text 
messaging allows for two-way communication between the 
school and home. Third, the unexpected communication 
among parents seemed to serve as a natural reinforcer for 
continued engagement in the home and community. 

In addition, parents responded positively to the use 
of video modeling, suggesting this strategy helped increase 
parents’ participation in the home program. These findings 
are congruent with Jabaley et al. (2011), who found video 
modeling an effective tool for changing parents’ behavior. 
These findings are illuminating, as they contribute to the 
body of literature supporting the use of video modeling as 
a way to provide instruction not only for children but also 
for adults. 

Although the social validity results were positive, the 
alternating treatments design did not reveal clear experimental 
effects associated with the home review component. This 
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was in part due to the fact that the listening center inter- 
vention alone is rather successful in teaching vocabulary 
words, and with only two words per book, there was limited 
opportunity to see differential effects. In addition, researchers 
observed a possible confound, as the teacher did not limit 
instruction to listening centers; she was observed review- 
ing the target words in the classroom. This may have aug- 
mented learning and contributed to ceiling effects. 


Experiment 2 (Sites 2 and 3) 


Despite the lack of clear home review effects in Ex- 
periment 1, parent feedback indicated that we were able 
to overcome many of the barriers to extending practice 
opportunities into the home. In Experiment 2, we sought 
to improve our ability to examine the effects of home review 
program in two ways. First, we used revised books that 
taught four words per book. The addition of two addi- 
tional words to each book was evaluated empirically before 
it was incorporated into this study (Peters-Sanders et al., 
2020). Notably, these additional words allowed for a 
more sensitive measure of differential effects. Second, we 
sought to enroll more participants from two new child- 
care settings. Because the social validity results from the 
previous iteration were positive, the other components of 
the home intervention remained the same, with the excep- 
tion of using a different, more secure application for 
school-home communication (i.e., Remind). A family de- 
mographic questionnaire was added to gather additional 
participant descriptive information (see Table 1). 


Participants 


Thirteen children and their parents were recruited 
from two different preschool sites, following IRB-approved 
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procedures. Demographic information is presented in 
Table 2. Despite both being in low-income urban areas, 
these centers differed considerably. All the parents in Site 2 
had college degrees, and the parents in Site 3 either had a 
high school diploma or had attended some college. Parents’ 
age ranged from 25 to 40 years. Six of eight mothers 

in Site 2 were married, one was single, and one was 
widowed. All five mothers in Site 3 were single; they were 
20-25 years old. 

Site 2 included eight children, and Site 3 included 
five children. In Site 2, six of the children were Caucasian, 
one child was African American, and one child was Latina. 
In Site 3, all five of the children were African American. 
The average age of children in Site 2 was 4;2 (range: 3;5— 
4;7 years old), and the average age in Site 3 was 4;1 (range: 
3;0-4;8 years old). The children in Site 2 had an average 
score of 95.4 (range: 79-108) on the CELF Preschool-2 
and an average score of 108.5 (range: 97-119) on the 
PPVT-4; two of six children scored more than 1 SD below 
the norm on the CELF Preschool-2. The children in Site 3 
had an average score of 81.2 (range: 71-92) on the CELF 
Preschool-2 and an average score of 93.8 (range: 88-101) 
on the PPVT-4; three of five children scored more than 
1 SD below the norm on the CELF Preschool-2. 

Families’ home literacy practices are summarized in 
Table 2. Across the two sites, most reported they read to their 
children three times a week. Most children used computers 
at home for approximately 1-3 hr a week. The majority 
of the parents in Site 2 stated that their children watched TV 
for about 1-2 hr a week, whereas the parents in Site 3 stated 
that their children watched TV for 3-6 hr a week. 


Measures 


Experiment 2 used the same language assessments, 
parent diary forms, and social validity interviews as Exper- 
iment 1. The Vocabulary Mastery Monitor was adapted to 
reflect the addition of two extra words. Instead of receiving 
a potential score of 4, this version had a maximum score 
of 8 per book (corresponding to each word receiving a po- 
tential of 2 points each). 

A family demographic questionnaire was incorpo- 
rated to gather additional descriptive information about 
home characteristics. The family questionnaire contained 
questions regarding the child’s ethnicity, language(s) spoken 
at home, how often stories were read at home, how often 
letters were reviewed at home, how often the computer was 
used at home, how frequently the children watched TV, 
whether the family or teacher had specific concerns, and 
the highest level of parental education. 


Procedure 


Consistent with the procedures of Study 1, parents 
were trained in person using the same format. All parents 
in Site 2 attended the in-person training held in a local 
library and were trained at the same time. In Site 3, two of 
the five parents were trained at the same time, and the others 


were trained individually in a quiet room at the preschool 
due to scheduling constraints. The same parent diary forms 
were sent home to track how often words were practiced at 
home. However, instead of home-school communication 
being maintained through WhatsApp, another freely avail- 
able application was used, Remind. Remind facilitated the 
scheduling of automated daily text reminders while also 
providing a platform to disseminate information and engage 
parents. 

The most significant modification made was increas- 
ing the number of words taught in each book from two to 
four words. Instead of sending one word home to review 
each week, children were given prompting materials for two 
of the four words taught at school each week. The words 
taught in each book are listed in Table 3. Because the 
words reviewed at home were increased to two words, the 
prompting materials were modified. The prompting mate- 
rials for the third iteration included 2 x 3 in. badges with 
a lanyard worn around children’s necks that included pic- 
tures representing the two target words in the front with 
the phrase “Ask me what these words mean?” and with the 
definitions of the words on the back. 


Fidelity and Reliability 


The implementation fidelity of 20% of the sessions 
of Experiment 2 was evaluated using the same procedural 
checklist previously described. The implementation fidelity 
averaged 99% (range: 94°%-100%). 

The fidelity of administration for the Vocabulary 
Mastery Monitor was calculated for 20% of the total test- 
ing sessions. The fidelity of assessment administration 
was 100%. The item-by-item response agreement on 20% 
of the Vocabulary Mastery Monitor averaged 99% (range: 
88%-100%). 


Results 


Parent Implementation and Social Validity 
of the Home Component 


Parent Implementation 

The data from parent diary forms indicated that 
there were site differences in parents’ implementation of 
the home program. In Site 2, all the parents returned 
100% of the parent diary forms for all six books. Parents 
reported practicing the words at home approximately 10— 
24 times a week. The parents in Site 2 were responsive in 
their communication through Remind. One parent shared 
pictures of her daughter practicing the words at home. Many 
parents left comments on the texting application and asked 
questions about how they could improve their children’s 
learning. For example, one parent commented, “I believe 
the word smart is too hard for my child. She doesn’t under- 
stand that abstract word. What do I do?” In contrast, 
none of the parents in Site 3 submitted the parent diary 
forms throughout the study. They did not respond to the 
researchers’ attempts for communication through Remind. 
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Social Validity 

The social validity results corroborated the results in 
the graphs and the parent diary forms. All parents in the 
Site 2 completed the social validity interview, providing 
detailed responses and feedback. Overall, all the parents 
in Site 2 shared their excitement about the program, stating 
that the intervention provided an effective means of having 
meaningful conversations with their children. Parents 
in Site 2 reported that they practiced the words at home 
every day, incorporating the use of these words into their 
daily routines. One parent reported, “We modeled the mean- 
ing of the words to help increase the understanding of the 
words. For example, we pretended to comfort the baby 
doll.” Parents suggested the program be improved by re- 
ducing the number of badges sent home each week and by 
sending home texts with ideas on how to use the words in 
conversation. Other parents stated they would have liked 
receiving additional suggestions for deepening children’s 
understanding of the words. 

Only three of five parents in Site 3 completed the 
social validity interview. Although the researchers attempted 
reaching the other parents at least three times via phone 
and by talking to the teacher, they did not respond. The 
teacher stated that she also had difficulty communicating 
with these parents because they were transitioning between 
jobs and because their phone service was terminated. Like 
the parents from Site 2, the parents in Site 3 noted they 
enjoyed the program. However, they reported they did not 
complete the parent diary forms because they were busy. 
One parent reported that it was difficult for her to imple- 
ment the home program because her child stayed with his 
grandmother, who did not speak English, while she worked 
two jobs. When asked how the home program could be 
improved, one parent recommended that the researchers 
meet with the parents in person more often, such as once a 
week or once a month. 


Differential Effects on Children’s 
Vocabulary Learning 


The results of Experiment 2 are shown in Figures 2 
and 3. Graphs are arranged by the estimated order of home 
implementation as indicated by how many times a week 
parents reported practicing the words at home through the 
parent diary forms (from highest to lowest, starting on the 
top left and progressing to the bottom right). The x-axis 
represents each book, and the y-axis presents children’s 
performance on the posttest Vocabulary Mastery Monitor 
probes. Because children had low scores at pretest (M = 0.06 
word points, SD = 0.14), only the posttest measures were 
graphed. The solid black lines with the black circles show 
children’s scores on the school-only words, and the dashed 
black lines with the open squares show children’s scores on 
words in the school + home condition. 

As can be seen in Figure 2, the dashed lines and 
squares tended to be equal or higher than the solid lines 
and circles. Among the eight children in Site 2, seven showed 
greater vocabulary learning in school + home condition. 
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Only Houston showed mixed effects, but with slightly 
higher learning in the school + home condition (M = 83.3% 
vs. 75%). Learning was consistently equal or greater for 
the school + home condition for Odell, Cynthia, Lola, 
Tommy, and Alex. Andrew and Annie had greater learn- 
ing in the school + home condition for four of six books, 
but greater learning in the school-only condition for one 
book each. 

In contrast to Site 2, children in Site 3 (see Figure 3) 
made smaller gains in vocabulary learning, and only Martin 
showed a clear advantage of the school + home condition, 
for four of five books. One child (Collin) only learned one 
word in six books. One child (Anthony) learned more words 
in the school-only condition for three of the six books. The 
two remaining children (Layla and Aliya) showed mixed 
effects. 

In summary, vocabulary learning was greater in the 
school + home condition for nine of the 13 children (69%), 
and only one of the 13 showed an advantage in the school- 
only condition. In Site 2, out of 51 total books, a school + 
home advantage was replicated for 32 books (63%) and a 
school only advantage was replicated for three books (6%). 
In Site 3, out of a total of 29 books, a school + home ad- 
vantage was replicated for 10 books (34%) and a school- 
only advantage was replicated for five books (17%). 

MLM analyses were conducted using the approach 
described in Experiment |. The results of the random coef- 
ficient models for Sites 2 and 3 were consistent with the 
results of the visual analysis (see Table 3). Linearity, homo- 
scedasticity, and normality assumptions were evaluated for 
each residual in the model. The results confirmed a homo- 
scedastic residual distribution with multivariate normality 
and linear effects for Sites 2 and 3. The results indicated 
that children from Site 2 learned an average of 0.66 word 
points per word in the school-only condition and an aver- 
age of 1.44 word points per word in the school + home 
condition. Thus, the school + home condition significantly 
expanded children’s word knowledge by an estimated 0.78 
word points, 7(179) = 6.15, p < .001. 

Children from Site 3 learned an estimated average of 
0.57 word points in the school-only condition and an aver- 
age of 0.65 word points in the school + home condition. 
The school + home condition helped these children to expand 
their word knowledge by an estimated 0.09 word points, 
which was not a statistically significant difference, (110) = 
0.54, p = .588. 

The results indicated that children in Site 2 experienced 
greater word learning in the school + home condition, indi- 
cating that the home review component enhanced learning 
effects. However, this effect was not observed in children 
in Site 3. As in Experiment 1, the model also revealed sig- 
nificant variance components for within-child variations of 
children from Sites 2 (6* = 0.75) and 3 (o* = 0.73). 


Discussion 


Consistent with the findings from Experiment 1, parents 
in Experiment 2 (Sites 2 and 3) responded positively to 
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Figure 2. Experiment 2 results (Site 2). Children’s posttest scores on the Vocabulary Mastery Monitor for each book. Open squares denote 
the home + school condition, and closed circles denote the school-only condition. Graphs are arranged by the level of home implementation 
as indicated by parent diary forms (highest to lowest, starting from the top left graph and progressing to the right). 
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various aspects of the home review program, including the 
in-person parent trainings, use of text messaging, and prompt- 
ing materials. However, the effects of the home program 
differed between Sites 2 and 3. Only the children in Site 2 
demonstrated robust gains in their vocabulary skills follow- 
ing the home review of Story Friends words. 

One possible reason for these site differences is that the 
children in Site 2 started out with higher language skills 
than the children in Site 3. Although they did not differ in 
their baseline scores for the words taught (averaging 2.9% 
and 2.5% of the words, respectively), children in Site 3 had 


lower expressive and receptive language test scores than 
children in Sites | and 2. Previous research on Story Friends 
and other vocabulary interventions have found that chil- 
dren with initially higher language skills are likely to learn 
more vocabulary words than children with lower vocabu- 
lary skills after receiving instruction (Coyne et al., 2004; 
Goldstein et al., 2016; Penno et al., 2002; Peters-Sanders 
et al., 2020). 

However, these differences in language skill do not 
fully explain the results. Children in Sites 1 and 3 had similar 
average scores on measures of their receptive vocabulary 
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Figure 3. Experiment 2 results (Site 3). Children’s posttest scores on the Vocabulary Mastery Monitor for each book. Open squares denote 
the home + school condition, and closed circles denote the school-only condition. Graphs are arranged by the level of home implementation 
as indicated by parent diary forms (highest to lowest, starting from the top left graph and progressing to the right). 


Anthony 


Aliyah 


Posttest score on Vocabulary Mastery Monitor 


but differed in their learning gains. Thus, learning differ- 
ences are more likely to be attributable to the level of parent 
engagement in these two sites. These differences beg the 
question “What made Site 3 unique?” There were demo- 
graphic characteristics that set Sites 2 and 3 apart. Most of 
the parents in Site 2 were older and had college degrees, 
and there were two adults in the home. Conversely, all of 
the caretakers in Site 3 were single, generally younger, and 
had a high school education. Although there is evidence 
that parents’ education is a strong predictor of children’s 
vocabulary skills (Hart & Risley, 1995), attributing levels 
of parent engagement solely to parents’ education and 
marital status is insufficient. Parents in Site 3 were similar 
to parents in Site 1. Both sets of parents were the sole care- 
takers and had received a high school education. However, 
while the parents in Site | were highly engaged, the parents 
in Site 3 did not respond to the researchers’ communication 
attempts, reporting that their busy schedules conflicted 
with their participation. 

Another notable difference between Sites 2 and 3 
that could have influenced the level of parent engagement 
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was TV watching at home. In Site 2, parents reported their 
children are watching TV with an average of 1-2 hr per 
week, whereas in Site 3, parents reported their children 
are watching TV with an average of 3-6 hr per week. Re- 
search suggests that the amount of background TV noise 
reduces the quantity and quality of parent-child inter- 
action (Christakis et al., 2009; Kirkorian et al., 2009; 
Mendelsohn et al., 2008). Kirkorian et al. (2009) warn that 
background noise containing speech can reduce parents’ 
responsiveness to children’s communicative attempts. 
There also is evidence suggesting this limited parent-child 
social interactions can delay children’s language devel- 
opment (Mendelsohn et al., 2008). Findings from this 
experiment provide only a preliminary indication suggest- 
ing that the number of hours children spend watching TV 
might predict parents’ engagement and responsiveness to 
the home program. Future studies should investigate the 
potential role of moderating factors (such as maternal 
education, hours of TV watching, and the number of envi- 
ronmental stressors) on parents’ participation in home 
programs. 
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Findings from Experiment 2 indicate that, when parents 
implement the home review of Story Friends words, their 
children learn more of the academic words. Despite these 
promising results, these findings also provide preliminary 
evidence that there are differences in the engagement of 
families in the home review program, likely associated with 
a number of risk factors. 


General Discussion 


Despite mounting evidence that children benefit from 
living in homes with rich conversations more likely to in- 
clude academic vocabulary, few studies directly evaluate 
the effects of bridging vocabulary instruction between the 
school and home. The purpose of this article was to de- 
scribe the initial steps in developing and evaluating a home 
component for a supplemental oral language curriculum 
for preschoolers. The effects of the home review program 
were evaluated by gathering social validity data and mea- 
suring children’s vocabulary gains. 


The Value of Social Validity 

The iterative nature of the development of this home 
review program highlights the utility of taking full advan- 
tage of social validity assessments when creating and refin- 
ing home-based interventions. First, social validity data 
provide a clearer picture of the studies’ results. Evaluating 
the effects of the home review program solely through 
children’s vocabulary learning provided an incomplete pic- 
ture of factors contributing to implementation in the home. 
The social validity measures revealed implementation dif- 
ferences and potential barriers to participation. Identifying 
these barriers to participation resulted in active problem 
solving that sought to address parents’ needs. Second, the 
social validity results provided important insights that 
helped explain why the home vocabulary program’s effects 
varied across sites. Third, incorporating social validity 
measures during the development process reinforced the 
notion that parents were an integral part of children’s word 
learning. In both experiments, the researchers asked the 
parents to provide them with feedback regarding the inter- 
vention materials at the beginning of the study. This feed- 
back was important in creating reciprocity between the 
parents and the researchers, seemingly enhancing the level 
of parent buy-in and engagement. 

Goldstein (2016) argues that social validity should 
not be an afterthought. Instead, social validity should be 
incorporated at all stages of intervention development. 
Despite the value of social validity, most interventions 
focus exclusively on establishing internal validity (Goldstein 
et al., 2014; Soto et al., 2019). For example, Soto et al. 
(2019) conducted a systematic review of 17 studies evaluat- 
ing the effects of early literacy interventions for young 
Latino children who were dual language learners. The re- 
sults of this review indicated that, although most studies 
demonstrated strong internal validity, only two studies 
incorporated social validity measures. Incorporating social 


validity measures bolster generalizing power (external 
validity) by ensuring that evidenced-based interventions 
are sustainable in real-world settings (Goldstein, 2016; 
Krathwohl, 2009). Social validity assessments hold po- 
tential to more nimbly adapt interventions to varying cul- 
tures, community circumstances, and needs of individual 
families. 


Promoting Parent Engagement 

During the inception of the home component of 
Story Friends, we hypothesized that developing a seemingly 
simple intervention incorporating the use of stickers with 
prompts (e.g., Ask me what brave means?) would result in 
high parent participation. However, the disappointing re- 
sults from the first iteration of the home program revealed 
that sending materials home without first establishing 
a positive, reciprocal relationship with parents or without 
explicitly explaining to parents how to use the materials 
was insufficient. 

The second and third iterations incorporated parent 
engagement strategies that produced more promising find- 
ings. These strategies included providing in-person train- 
ings, using video modeling, sending daily text reminders, 
maintaining two-way communication, and addressing po- 
tential participation barriers. Improved vocabulary learn- 
ing when these strategies were applied demonstrates the 
importance of developing home programs that adapt to 
families’ needs. The findings from these iterations illustrate 
the importance of evaluating and addressing the barriers 
parents might experience that preclude them from partici- 
pating in home programs. 


Variability in Parent Engagement 
and Home Implementation 

Although not surprising, the variability in parents’ 
level of participation and engagement was striking. Predict- 
ably, effects of vocabulary instruction and home review 
activities were weakest in homes reporting low home im- 
plementation. The parents with the lowest reported imple- 
mentation of the home program noted that, although they 
saw the value of the program, their hectic schedules pre- 
cluded their full participation. These findings are elucidating 
as they reveal that, even if parents believe an educational 
program to be beneficial and attempts are made to reduce 
their participation barriers, some parents may not complete 
interventions at home. 

Despite our efforts to encourage parents’ feedback, 
we most likely overlooked their actual needs, such as food 
security, rent payments, or behavior health counseling. It 
is likely that some parents were experiencing extenuating 
barriers (e.g., financial, social-emotional) precluding their 
successful implementation of the home program. Therefore, 
they might have benefitted from being referred to other 
professionals (e.g., social workers) before participating in 
the home review program. If our goal is to reach the fami- 
lies with the most compounding stress, it is vital for future 
studies to identify and address potential moderating 
factors (financial, socioemotional, home organization, TV 
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watching, etc.) on parents’ implementation of home-based 
interventions. 


Limitations and Future Directions 


These iterative studies have limitations warranting 
consideration in future studies. These experiments were 
conducted with a small sample of classrooms and fami- 
lies. These promising results may warrant additional 
systematic replications and larger-scale group design 
experiments. 

Another limitation is the limited data on children’s 
language skills and parents’ demographic information, 
particularly for the pilot study and Experiment 1. This in- 
formation was not gathered in the first two iterations be- 
cause the initial focus was to evaluate the feasibility of the 
home review activities; however, collecting these data 
would have strengthened these studies. Moreover, in the 
first experiment, the teacher’s involvement in the classroom 
might have confounded the results of the study. It is likely 
that her review of words outside the home extension program 
conflated the school-only condition effects. 

Alternating treatments designs are not sensitive to 
small differences in learning, especially when treatment 
conditions overlap. Multiple treatment interference is a 
common threat to the interpretation of results using this 
design. Relying on counterbalancing of so few words (two 
in Experiment | and four in Experiment 2) also limits the 
opportunity to detect learning differences. Although the 
design offers multiple opportunities to evaluate learning 
within and among participants, an inability to gather 
objective, in situ data on home practice limits our ability 
to directly relate learning to our independent variable of 
interest. 

It is difficult to capture how often parents actually 
practice words at home. Although data gathered from 
the parent diary forms provided an indication of the level 
of parent participation in the home program, not all parents 
returned the diary forms on a weekly basis. When inter- 
viewed, parents also indicated that, even when they returned 
the parent diary forms, they were a rough estimate of tar- 
geted vocabulary use in the home. Moreover, a failure to 
return diaries did not necessarily mean that those parents 
did not review words at home. Future studies should con- 
sider better ways of capturing parents’ use of targeted, aca- 
demic vocabulary words at home. Including the use of the 
Language Environment Analysis system (LENA Founda- 
tion) might help gather insightful information regarding 
home vocabulary practices. 

Because the home program was an extension of an 
existing vocabulary curriculum, Story Friends, only words 
preselected by the research team were reviewed at home. 
One benefit of this approach was that we sent home child- 
friendly definitions and academic vocabulary words that 
parents may not readily use in their daily routines. Our 
hope is that it encourages parents to target additional aca- 
demic words. Future studies should determine whether the 
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vocabulary words reviewed at home are frequently used by 
parents in their daily routines. 

In this study, the only vocabulary measures used to 
determine treatment effects were the research-developed 
curriculum-based Vocabulary Mastery Monitors. The use 
of curriculum-based measures provides a robust assessment 
of the taught words and is often a better measure of learn- 
ing than norm-referenced assessments that only measure 
generalized language abilities (National Institute of Child 
Health and Human Development, 2000). Often, general- 
ized language assessments do not assess children’s knowl- 
edge of sophisticated vocabulary words such as the ones 
from Story Friends, and because of this, we typically do 
not see significant growth on these measures as a result of 
our intervention. However, improved outcomes on the 
Vocabulary Mastery Monitors indicate that children are 
adding new vocabulary terms to their lexicon. Learning 
these new words may help strengthen their semantic net- 
works, enhance their word finding abilities, and improve 
recall of words and their meanings. Future studies should 
include generalization measures of vocabulary to determine 
the extent to which learning a finite number of challenging 
words influences overall vocabulary knowledge. 

Large-scale studies have the potential to elucidate 
the influence of moderating factors (number of hours TV 
watched, number of adults in the home, number of chil- 
dren, level of parents’ education, etc.) on parent implemen- 
tation of home-based interventions and child learning 
effects. For many families, it might be beneficial to provide 
resources and supports to help create homes conducive to 
child growth and development generally, in tandem with 
introducing families to specific home language interventions. 

Researchers were the ones communicating with 
parents and facilitating home implementation. Future studies 
should study the effects when teachers or clinicians are 
teaching and encouraging parents to practice new vocabu- 
lary words. Furthermore, it is important to consider barriers 
to parents’ participation, such as access to smart phones or 
funds for sending text messages and images. Conducting 
in-person trainings also presents challenges as one strives 
to accommodate parents’ disparate work schedules. Future 
studies should investigate the potential benefit of using 
videoconferencing to overcome these challenges, while being 
mindful of the need to ensure that all parents have equita- 
ble access to all aspects of home programming. 


Conclusion 


This article highlights the role of home review activities 
on enhancing language development of at-risk preschoolers 
within early childhood education settings. Despite the 
strong evidence that young children benefit from home 
environments that provide rich contexts for vocabulary 
learning, there is limited empirical research on effective 
strategies for promoting parent engagement to increase the 
vocabulary instruction occurring at school. These studies 
provide preliminary evidence of effective ways of aug- 
menting children’s learning of vocabulary skills through 
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parent-focused home interventions. The findings showcase 
the utility of establishing two-way communication between 
school and home, addressing potential barriers to partici- 
pation, and nurturing positive relationships with parents. 
Nevertheless, there continues to be a tremendous need for 
future work to evaluate alternative means of strengthening 
the home language environment if we are to close the 

30 million word gap too often experienced by children living 
in families with compounding risk factors (Suskind & 
Suskind, 2015). 
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